Overlay size option: --overlay2=all:memory,size=1g #11723

stepancheg · 2025-05-13T06:04:10Z

Implement #11504.

Add optional size= parameter to --overlay2 flag: when it is set, overlay upper layer is mounted with given size flag.

Tested:

sudo bazel-bin/runsc/runsc_/runsc --overlay2=all:memory,size=1m run --bundle ~/bu cont-i

root@:/# dd if=/dev/urandom of=/x bs=1k count=1000
1000+0 records in
1000+0 records out
1024000 bytes (1.0 MB, 1000 KiB) copied, 0.0109061 s, 93.9 MB/s

root@:/# dd if=/dev/urandom of=/x bs=1k count=1100
dd: error writing '/x': No space left on device
1021+0 records in
1020+0 records out
1044480 bytes (1.0 MB, 1020 KiB) copied, 0.011396 s, 91.7 MB/s

  -overlay
        DEPRECATED: use --overlay2=all:memory to achieve the same effect
  -overlay2 value
        wrap mounts with overlayfs. Format is
        * 'none' to turn overlay mode off
        * {mount}:{medium}[size={size}], where
            'mount' can be 'root' or 'all'
            'medium' can be 'memory', 'self' or 'dir=/abs/dir/path' in which filestore will be created
            'size' optional parameter overrides default overlay upper layer size
         (default root:self)
  -panic-log string
        file path where panic reports and other Go's runtime messages are written.

stepancheg · 2025-05-15T20:43:42Z

Changed syntax from

--overlay2=all:memory:size=1g

to

--overlay2=all:memory,size=1g

ayushr2

Please see the discussion in #11504. I'd prefer not to complicate the --overlay2 flag any further. If you really want to use memory backend via --overlay2=*:memory, then consider applying memory limits to the entire container. OR try using the file backend with self medium and setting up size limits on the host rootfs directory.

runsc/boot/vfs.go

stepancheg · 2025-05-15T23:40:43Z

(deleted wrong comments; let me think)

stepancheg · 2025-05-16T02:03:39Z

New version where overlay size is passed in GoferMountConf.UpperSize.

runsc/boot/gofer_conf.go

runsc/boot/vfs.go

runsc/container/container.go

stepancheg · 2025-05-21T15:56:28Z

Next version ready for review.

stepancheg · 2025-05-28T16:33:21Z

Hello, can I do something to have this PR accepted?

ayushr2

LGTM, thanks!

runsc/boot/mount_hints.go

ayushr2 · 2025-05-29T06:40:42Z

Also, could you update the docs here:

gvisor/g3doc/user_guide/filesystem.md

Line 37 in 33d4ca3

### Root Filesystem Overlay

I think this is useful and others who want to set disk limits without having to jump all the hoops of reconfiguring their Docker home dir with a different filesystem etc will find such docs useful.

stepancheg · 2025-05-29T21:11:57Z

Done and done.

stepancheg · 2025-06-03T12:13:26Z

Google internal checks FAILED

Can I help with it?

ayushr2 · 2025-06-03T16:19:01Z

Our presubmits were broken for past few days. Triggering submit again.

Implement #11504. Add optional `size=` parameter to `--overlay2` flag: when it is set, overlay upper layer is mounted with given size flag. Tested: ``` sudo bazel-bin/runsc/runsc_/runsc --overlay2=all:memory,size=1m run --bundle ~/bu cont-i root@:/# dd if=/dev/urandom of=/x bs=1k count=1000 1000+0 records in 1000+0 records out 1024000 bytes (1.0 MB, 1000 KiB) copied, 0.0109061 s, 93.9 MB/s root@:/# dd if=/dev/urandom of=/x bs=1k count=1100 dd: error writing '/x': No space left on device 1021+0 records in 1020+0 records out 1044480 bytes (1.0 MB, 1020 KiB) copied, 0.011396 s, 91.7 MB/s ``` ``` -overlay DEPRECATED: use --overlay2=all:memory to achieve the same effect -overlay2 value wrap mounts with overlayfs. Format is * 'none' to turn overlay mode off * {mount}:{medium}[size={size}], where 'mount' can be 'root' or 'all' 'medium' can be 'memory', 'self' or 'dir=/abs/dir/path' in which filestore will be created 'size' optional parameter overrides default overlay upper layer size (default root:self) -panic-log string file path where panic reports and other Go's runtime messages are written. ``` FUTURE_COPYBARA_INTEGRATE_REVIEW=#11723 from stepancheg:overlay-size 451544b PiperOrigin-RevId: 764885043

runsc/boot/gofer_conf_test.go

runsc/config/config_test.go

Implement #11504. Add optional `size=` parameter to `--overlay2` flag: when it is set, overlay upper layer is mounted with given size flag. Tested: ``` sudo bazel-bin/runsc/runsc_/runsc --overlay2=all:memory,size=1m run --bundle ~/bu cont-i root@:/# dd if=/dev/urandom of=/x bs=1k count=1000 1000+0 records in 1000+0 records out 1024000 bytes (1.0 MB, 1000 KiB) copied, 0.0109061 s, 93.9 MB/s root@:/# dd if=/dev/urandom of=/x bs=1k count=1100 dd: error writing '/x': No space left on device 1021+0 records in 1020+0 records out 1044480 bytes (1.0 MB, 1020 KiB) copied, 0.011396 s, 91.7 MB/s ``` ``` -overlay DEPRECATED: use --overlay2=all:memory to achieve the same effect -overlay2 value wrap mounts with overlayfs. Format is * 'none' to turn overlay mode off * {mount}:{medium}[size={size}], where 'mount' can be 'root' or 'all' 'medium' can be 'memory', 'self' or 'dir=/abs/dir/path' in which filestore will be created 'size' optional parameter overrides default overlay upper layer size (default root:self) -panic-log string file path where panic reports and other Go's runtime messages are written. ``` FUTURE_COPYBARA_INTEGRATE_REVIEW=#11723 from stepancheg:overlay-size 451544b PiperOrigin-RevId: 764885043

stepancheg · 2025-06-03T17:40:29Z

Fixed tests, replaced %d with %s.

Sorry I missed the tests: I cannot run every test on my machine

bazel test runsc/...

fails with error I don't understand.

pkg/sentry/socket/plugin/stack/notifier.go:114:21: undefined: cgo.EpollCreate
pkg/sentry/socket/plugin/stack/notifier.go:120:14: undefined: cgo.EpollWait
pkg/sentry/socket/plugin/stack/notifier.go:155:7: undefined: cgo.EpollCtl
pkg/sentry/socket/plugin/stack/notifier.go:158:7: undefined: cgo.EpollCtl
pkg/sentry/socket/plugin/stack/notifier.go:162:7: undefined: cgo.EpollCtl
pkg/sentry/socket/plugin/stack/provider.go:74:16: undefined: cgo.Socket
pkg/sentry/socket/plugin/stack/readwriter.go:61:28: undefined: cgo.Recvmsg
pkg/sentry/socket/plugin/stack/readwriter.go:73:12: undefined: cgo.Read
pkg/sentry/socket/plugin/stack/readwriter.go:75:12: undefined: cgo.Readv
pkg/sentry/socket/plugin/stack/readwriter.go:91:13: undefined: cgo.Sendto
pkg/sentry/socket/plugin/stack/readwriter.go:91:13: too many errors

Implement #11504. Add optional `size=` parameter to `--overlay2` flag: when it is set, overlay upper layer is mounted with given size flag. Tested: ``` sudo bazel-bin/runsc/runsc_/runsc --overlay2=all:memory,size=1m run --bundle ~/bu cont-i root@:/# dd if=/dev/urandom of=/x bs=1k count=1000 1000+0 records in 1000+0 records out 1024000 bytes (1.0 MB, 1000 KiB) copied, 0.0109061 s, 93.9 MB/s root@:/# dd if=/dev/urandom of=/x bs=1k count=1100 dd: error writing '/x': No space left on device 1021+0 records in 1020+0 records out 1044480 bytes (1.0 MB, 1020 KiB) copied, 0.011396 s, 91.7 MB/s ``` ``` -overlay DEPRECATED: use --overlay2=all:memory to achieve the same effect -overlay2 value wrap mounts with overlayfs. Format is * 'none' to turn overlay mode off * {mount}:{medium}[size={size}], where 'mount' can be 'root' or 'all' 'medium' can be 'memory', 'self' or 'dir=/abs/dir/path' in which filestore will be created 'size' optional parameter overrides default overlay upper layer size (default root:self) -panic-log string file path where panic reports and other Go's runtime messages are written. ``` FUTURE_COPYBARA_INTEGRATE_REVIEW=#11723 from stepancheg:overlay-size 451544b PiperOrigin-RevId: 764885043

ayushr2 · 2025-06-03T19:18:43Z

I think you can run tests using make test TARGETS=runsc/config:config_test, it uses bazel from our bazel container.

Implement #11504. Add optional `size=` parameter to `--overlay2` flag: when it is set, overlay upper layer is mounted with given size flag. Tested: ``` sudo bazel-bin/runsc/runsc_/runsc --overlay2=all:memory,size=1m run --bundle ~/bu cont-i root@:/# dd if=/dev/urandom of=/x bs=1k count=1000 1000+0 records in 1000+0 records out 1024000 bytes (1.0 MB, 1000 KiB) copied, 0.0109061 s, 93.9 MB/s root@:/# dd if=/dev/urandom of=/x bs=1k count=1100 dd: error writing '/x': No space left on device 1021+0 records in 1020+0 records out 1044480 bytes (1.0 MB, 1020 KiB) copied, 0.011396 s, 91.7 MB/s ``` ``` -overlay DEPRECATED: use --overlay2=all:memory to achieve the same effect -overlay2 value wrap mounts with overlayfs. Format is * 'none' to turn overlay mode off * {mount}:{medium}[size={size}], where 'mount' can be 'root' or 'all' 'medium' can be 'memory', 'self' or 'dir=/abs/dir/path' in which filestore will be created 'size' optional parameter overrides default overlay upper layer size (default root:self) -panic-log string file path where panic reports and other Go's runtime messages are written. ``` FUTURE_COPYBARA_INTEGRATE_REVIEW=#11723 from stepancheg:overlay-size 32c91c3 PiperOrigin-RevId: 764885043

stepancheg · 2025-06-03T19:57:07Z

runsc/config:config_test

works without container, but if I run in container with make test TARGETS=runsc/..., the error is the same.

Anyway, I will run

bazel test runsc/boot/... runsc/config/... runsc/container/...

for time being.

Implement #11504. Add optional `size=` parameter to `--overlay2` flag: when it is set, overlay upper layer is mounted with given size flag. Tested: ``` sudo bazel-bin/runsc/runsc_/runsc --overlay2=all:memory,size=1m run --bundle ~/bu cont-i root@:/# dd if=/dev/urandom of=/x bs=1k count=1000 1000+0 records in 1000+0 records out 1024000 bytes (1.0 MB, 1000 KiB) copied, 0.0109061 s, 93.9 MB/s root@:/# dd if=/dev/urandom of=/x bs=1k count=1100 dd: error writing '/x': No space left on device 1021+0 records in 1020+0 records out 1044480 bytes (1.0 MB, 1020 KiB) copied, 0.011396 s, 91.7 MB/s ``` ``` -overlay DEPRECATED: use --overlay2=all:memory to achieve the same effect -overlay2 value wrap mounts with overlayfs. Format is * 'none' to turn overlay mode off * {mount}:{medium}[size={size}], where 'mount' can be 'root' or 'all' 'medium' can be 'memory', 'self' or 'dir=/abs/dir/path' in which filestore will be created 'size' optional parameter overrides default overlay upper layer size (default root:self) -panic-log string file path where panic reports and other Go's runtime messages are written. ``` FUTURE_COPYBARA_INTEGRATE_REVIEW=#11723 from stepancheg:overlay-size 32c91c3 PiperOrigin-RevId: 764885043

stepancheg force-pushed the overlay-size branch 4 times, most recently from c25a5c4 to 10ff226 Compare May 13, 2025 06:20

stepancheg marked this pull request as ready for review May 13, 2025 06:20

stepancheg force-pushed the overlay-size branch from 10ff226 to 1390765 Compare May 15, 2025 20:42

stepancheg changed the title ~~Overlay size option: --overlay2=all:memory:size=1g~~ Overlay size option: --overlay2=all:memory,size=1g May 15, 2025

stepancheg force-pushed the overlay-size branch 2 times, most recently from 9642db8 to d7cb29a Compare May 15, 2025 20:50

ayushr2 reviewed May 15, 2025

View reviewed changes

runsc/boot/vfs.go Outdated Show resolved Hide resolved

ayushr2 mentioned this pull request May 15, 2025

Overlay size, as alternative to --storage-opt size=30G #11504

Open

stepancheg mentioned this pull request May 15, 2025

Newtype OverlayMedium #11735

Closed

stepancheg force-pushed the overlay-size branch from d7cb29a to 5b8bf6b Compare May 16, 2025 02:02

ayushr2 reviewed May 19, 2025

View reviewed changes

stepancheg force-pushed the overlay-size branch 2 times, most recently from 1fff6f9 to 9cd19fd Compare May 21, 2025 15:54

stepancheg requested a review from ayushr2 May 21, 2025 15:56

ayushr2 reviewed May 29, 2025

View reviewed changes

runsc/boot/mount_hints.go Outdated Show resolved Hide resolved

stepancheg force-pushed the overlay-size branch from 9cd19fd to 451544b Compare May 29, 2025 21:11

ayushr2 approved these changes May 29, 2025

View reviewed changes

ayushr2 added the ready to pull label May 29, 2025

copybara-service bot mentioned this pull request Jun 3, 2025

Overlay size option: --overlay2=all:memory,size=1g #11784

Merged

ayushr2 reviewed Jun 3, 2025

View reviewed changes

runsc/boot/gofer_conf_test.go Show resolved Hide resolved

runsc/config/config_test.go Show resolved Hide resolved

Option --overlay-size

32c91c3

stepancheg force-pushed the overlay-size branch from 451544b to 32c91c3 Compare June 3, 2025 17:36

copybara-service bot merged commit b14059d into google:master Jun 4, 2025
3 checks passed

Overlay size option: --overlay2=all:memory,size=1g #11723

Overlay size option: --overlay2=all:memory,size=1g #11723

Uh oh!

Conversation

stepancheg commented May 13, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

stepancheg commented May 15, 2025

Uh oh!

ayushr2 left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

stepancheg commented May 15, 2025

Uh oh!

stepancheg commented May 16, 2025

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

stepancheg commented May 21, 2025

Uh oh!

stepancheg commented May 28, 2025

Uh oh!

ayushr2 left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

ayushr2 commented May 29, 2025

Uh oh!

stepancheg commented May 29, 2025

Uh oh!

stepancheg commented Jun 3, 2025

Uh oh!

ayushr2 commented Jun 3, 2025

Uh oh!

Uh oh!

Uh oh!

stepancheg commented Jun 3, 2025

Uh oh!

ayushr2 commented Jun 3, 2025

Uh oh!

stepancheg commented Jun 3, 2025

Uh oh!

Uh oh!

Uh oh!

stepancheg commented May 13, 2025 •

edited

Loading